Automated relationship categorizer and visualizer

ABSTRACT

Aspects differentially drive the graphic display of links overlaid between people in a group photograph as a function of associated relationship type. Individuals are distinguished within the photograph and relationships are identified among the individuals that differ from one another with respect to type of relationship, by extracting relationship data via natural language processing relative to social network data of one or more of the identified individuals. Thus, a graphic display device is driven to display the identified relationships via each of different overlay elements that are depicted directly on the photograph, link respective ones of the identified individuals, and use different graphic elements to convey differences in respective types of the relationships that are determined among the identified individuals.

BACKGROUND

Social network sites such as Facebook®, Google+®, Twitter®, LinkedIn®,etc. provide a variety of information to users, including the display ofphotographs. (FACEBOOK is a trademark of the Facebook Corporation in theUnited States or other countries; GOOGLE PLUS is a trademark of theGoogle Corporation in the United States or other countries, TWITTER is atrademark of the Twitter Corporation in the United States or othercountries; LINKEDIN is a trademark of the LinkedIn Corporation in theUnited States or other countries.) Some sites enable users to tag otherusers that appear in photographs, using the photographs as key indexesto a variety of data associated with the other user, for example linkingactivity in the different user accounts.

SUMMARY

In one aspect of the present invention, a method for differentiallydriving a graphic display of links overlaid between people in a groupphotograph as a function of associated relationship type includes aprocessor identifying each of a plurality of different individuals thatare each visible within a photograph and distinguished from other onesof the visible individuals. A plurality of relationships are identifiedamong the identified individuals that differ from one another withrespect to type of relationship, by extracting relationship data vianatural language processing relative to social network data of at leastone of the identified individuals. Thus, a graphic display device isdriven to display the identified relationships via each of differentoverlay elements that are depicted directly on the photograph, linkrespective ones of the identified individuals, and use different graphicelements to convey differences in respective types of the relationshipsthat are determined among the identified individuals.

In another aspect, a system has a hardware processor in circuitcommunication with a computer readable memory and a computer-readablestorage medium having program instructions stored thereon. The processorexecutes the program instructions stored on the computer-readablestorage medium via the computer readable memory and thereby identifieseach of a plurality of different individuals that are each visiblewithin a photograph and distinguished from other ones of the visibleindividuals. A plurality of relationships are identified among theidentified individuals that differ from one another with respect to typeof relationship, by extracting relationship data via natural languageprocessing relative to social network data of at least one of theidentified individuals. Thus, a graphic display device is driven todisplay the identified relationships via each of different overlayelements that are depicted directly on the photograph, link respectiveones of the identified individuals, and use different graphic elementsto convey differences in respective types of the relationships that aredetermined among the identified individuals.

In another aspect, a computer program product for differentially drivinga graphic display of links overlaid between people in a group photographas a function of associated relationship type has a computer-readablestorage medium with computer readable program code embodied therewith.The computer readable program code includes instructions for executionwhich cause the processor to identify each of a plurality of differentindividuals that are each visible within the photograph anddistinguished from other ones of the visible individuals. A plurality ofrelationships are identified among the identified individuals thatdiffer from one another with respect to type of relationship, byextracting relationship data via natural language processing relative tosocial network data of at least one of the identified individuals. Thus,a graphic display device is driven to display the identifiedrelationships via each of different overlay elements that are depicteddirectly on the photograph, link respective ones of the identifiedindividuals, and use different graphic elements to convey differences inrespective types of the relationships that are determined among theidentified individuals.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of embodiments of the present invention will bemore readily understood from the following detailed description of thevarious aspects of the invention taken in conjunction with theaccompanying drawings in which:

FIG. 1 depicts a cloud computing node according to an embodiment of thepresent invention.

FIG. 2 depicts a cloud computing environment according to an embodimentof the present invention.

FIG. 3 depicts abstraction model layers according to an embodiment ofthe present invention.

FIG. 4 is a flow chart illustration of a method or process according toan embodiment of the present invention for.

FIG. 5 is a graphic illustration of an example of a driven display of agraphic device according to an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

FIG. 1 is a schematic of an example of a programmable deviceimplementation 10 according to an aspect of the present invention, whichmay function as a cloud computing node within a cloud computingenvironment. Programmable device implementation 10 is only one exampleof a suitable implementation and is not intended to suggest anylimitation as to the scope of use or functionality of embodiments of theinvention described herein. Regardless, programmable deviceimplementation 10 is capable of being implemented and/or performing anyof the functionality set forth hereinabove.

A computer system/server 12 is operational with numerous other generalpurpose or special purpose computing system environments orconfigurations. Examples of well-known computing systems, environments,and/or configurations that may be suitable for use with computersystem/server 12 include, but are not limited to, personal computersystems, server computer systems, thin clients, thick clients, hand-heldor laptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputer systems, mainframe computer systems, and distributed cloudcomputing environments that include any of the above systems or devices,and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 12 may be practiced in distributed cloudcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed cloud computing environment, program modules may be locatedin both local and remote computer system storage media including memorystorage devices.

The computer system/server 12 is shown in the form of a general-purposecomputing device. The components of computer system/server 12 mayinclude, but are not limited to, one or more processors or processingunits 16, a system memory 28, and a bus 18 that couples various systemcomponents including system memory 28 to processor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 2, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 2 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 3, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 2) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 2 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and processing for differentially driving agraphic display of links overlaid between people in a group photographas a function of associated relationship type.

FIG. 4 (or “FIG. 4”) illustrates a computer implemented method orprocess of an aspect of the present invention for differentially drivinga graphic display of links overlaid between people in a group photograph(or “photos,” pictures, graphic images, etc.) as a function ofassociated relationship type. A processor (for example, a centralprocessing unit (CPU)) executes code, such as code installed on astorage device in communication with the processor. At 102, in responseto an input (publication) of a photograph on a social networking websiteengaged by a first user via the processor (for example, via executing onan application associated with a log in ID of the first user), theprocessor identifies each of a plurality of different individuals thatare each visible within the photograph and distinguished from other onesof the visible individuals.

The identification at 102 is generally performed by one or more ofexecuting a facial recognition process, and parsing text identifierssaved to metadata of the photograph, such as tag data previously appliedto the photograph and saved with photograph metadata, or other types oftext data. In some aspects the user opens the photograph within anapplication executing on a processor that enables relationshipfunctionality through labels or tags applied by the user, for example adashboard that enables tagging via pulldown selections or text entryinto field boxes for application or search.

At 104 the processor identifies a plurality of relationships among theidentified individuals that differ from one another with respect to typeof relationship. This is accomplished by extracting relationship datarelative to the identified individuals via Natural Language Processingfrom a variety of social network and organizational data that includes:(i) a business association or organization data (company hierarchy,customer list, etc.) that incorporates or is associated with the firstuser or one or more of the identified individuals; (ii) family data ofthe first user or identified individuals; and (iii) social network ormedia friends and contact data of the first user or the identifiedindividuals (LINKEDIN connections, FACEBOOK friends, GOOGLE PLUS circlemembers, etc.). Some aspect may also include and consider (iv)recreational activity organization data relevant to a membership,activity or recreational activity of the first user or the identifiedindividuals (for example, a sport team roster that the first user oridentified individual belongs to or follows or supports); and/or (v)geographic location data of the photograph or a current location orresidence of the first user may also be considered, to help identifyindividuals or likely relationships (for example, the location of asporting event that includes the sport team as a participant). Stillother data sources will be apparent to one skilled in the art.

Annotations or tags of identified individuals within the photographindicate relationship data on an associated or appropriate socialnetwork site, and Natural Language Processing (NLP) is used to extractthe relationship data and define relationships at 104. Pre-configurationsettings may define streaming data sources for real-time relationshipdata. Illustrative but not limiting or exhaustive examples includegenealogy sites, business networks (LINKEDIN, etc.) and organizationstructures obtained from web sites of organizations, sporting team, etc.Relationships among the individuals may be automatically determined as afunction of metadata associated with the photograph and displayed to theuser.

Aspects incorporate dynamic identification of users within photographsfrom the use of facial recognition or face tagging. When a groupphotograph is published on a social network and accessed by the user,aspects may update facial recognition of the group photographs in thebackground at 104 in response to current and most recent taggingactivity, without intervention or request by the user. Thus, streamingdata sources may be used to obtain the relationship data in real-timefor the users identified in the group photo. If existing photos postedon a social site have annotations with relationship data, aspects mayuse natural language processing to extract the relationship data anddefine relationship in current photo being analyzed.

At 106, the processor iteratively repeats the steps of identifyingindividuals and their relationships relative to each other and the firstuser at 102 or 104, with each subsequent iteration benefiting from theuse and application of additional relationship data identified from aprevious iteration at 104, until no more individuals or relationshipsare identified during a last iteration of 102 and 104. This iterativeanalysis at 106 may be triggered in response to new tagging data inputsfrom streaming data sources, or triggered by a periodic prompt (forexample, refreshing the data every hour, day, or other set period).Thus, an identification of a first individual in the photograph as ateammate or co-worker of the first user in a previous iteration at 104may result in identification of another, second individual in thephotograph as a business associate or customer of the first individualas a function of applying business organization data associated with thefirst individual in a second iteration at 104.

At 108 the processor drives a display device to display the identifiedplurality of relationships determined at 104 via each of a plurality ofdifferent overlay elements that are depicted directly on the photograph,overlaid on the identified individuals, link respective ones of theidentified individuals, and use different graphic elements to conveydifference in respective types of the relationships determined among theidentified individuals. More particularly, the overlay elements arevisually distinguished from one another based on type of relationship,for example by different colors, font, line weights, and framing devicesused to link the identified individuals. The visually indicateddifferences quickly and readily convey to the user the different typesand bases on respective ones of the business association/organizationdata, family data, social network data, recreational activity,organization data and geographic location data. The overlay elements mayalso indicate where no relationship is found between respectiveidentified individuals.

The visualization of relationships in aspects of the present inventionenables different visual depictions and filter rules, depending oncontext of interest of a person viewing of the photograph of the viewinguser, such as family, business, social, sporting team, etc. Data feedsused to define the relationships (for example, at 104, FIG. 4) can bestatically defined or dynamic from streaming data source such asorganizational structure, genealogy, other photos, via annotation anduse of NLP.

In some aspects the display at 108 filters the overlay elements depictedbased on security, privacy or preference settings of the first user (orof the device incorporating the driven display. For example, in responseto recognizing a current public setting of the display device, or aviewing of the photograph by a family member, business relationshipoverlays may be suppressed or otherwise omitted from display at 108,wherein the display depicts only family and friend relationships, oronly family relationships, or the identified individuals. In suchaspects the first user may define who can view which overlayrelationships, and in which of different contexts.

Social network sites enable users to tag other users that appear inphotographs, and thereby use the photographs as key indexes linking avariety of data associated with the other user, for example linkingactivity in the different user accounts. However, the usefulness of suchtagging is limited within such sites with respect to other users who maynot be linked to or otherwise cognizant of some of the tagged users, andthereby may not be informed of the tags or otherwise informed ofrelationships between different people in the photos. Aspects of thepresent invention provide methods and systems that analyze and displaythe relationship of people in a group photo posted on a social networkto others who may not be directly linked to or otherwise cognizant ofone or more of the depicted people, thereby enhancing the socialnetworking capabilities of an individual site. The display of thisinformation may be selective and dependent on the nature of theconnection (personal, family, friends, business, etc.) relative topermissions for a viewer (second user) viewing the photograph.

Some aspects use understanding of the relationships determined forpeople in a group photo to perform targeted advertising of goods andservices. Cross-selling may be a natural extension of targetedadvertising, and based on key data points determined relative to theusers and individuals identified in the photos, such as birthdays,holidays, sporting event records (big game wins), etc.

Social networking sites may enable users to define family relationships,for example through a pull down menu of options for tags to apply to aphotograph (sister, brother, father, grandmother, cousin, aunt, uncle,etc.). From multiple places group photographs may be captured and sharedvia one or more social networking sites. However, if a friend of theuser views the photograph, they may not be aware or understand therelationships of the people in the photograph. Aspects provide formethods and systems by which a user can visualize the relationships ofpeople in the group photograph to provide greater context andunderstanding to extended users. For example, other social network usersmay provide comments or specific data relevant to a relationship, viaaccessing the photograph on a social network application used by thefirst user to acquire or display the photograph (at 102, FIG. 4).

Aspects use specific and informative linking overlays to directly linkpeople in a group photo with graphic elements that clearly convey thenature of relationships between the people, wherein the comments may begrouped and selectively displayed based on relationship type relative toa view status. FIG. 5 provides an illustration of a view of a groupphotograph 502 that is published to a social networking site, wherein auser has the option to enable the display of different relationshipsbetween individuals that are identified within the photograph 502 (asdetermined at 104, FIG. 4). The relationships are each depicted by aplurality of different respective overlay elements that each describesdifferent relationships between respective indicated pairs of theindividuals.

More particularly, a “Friends” 520 overlay links individuals 504 and 506and indicates that they are friends by the use of a text label 521“Friends” and triangular frames 511 that are disposed about or onto eachperson 504 and 506 at respective ends of the Friends 520 overlay.

A “Spouses” 526 overlay links individuals 504 and 508 and indicates thatthe type of their relationship is “family” by the linking to hexagonalframes 503 that are disposed about or onto each person 504 and 508. Thedifferent shapes of the frames 511 and 503 quickly convey differences inthe relationship type (friends vs family). The relationship subtypewithin the genus of “family,” here “spouses” is conveyed by the textlabel 523 “Spouses.”

Other family overlay links Brothers 524 and Brothers-in-law 522 alsoconvey their relationship is “family” by the linking to the hexagonalframes 503 that are disposed about or onto each respective person 508,510 and 512. The different relationship subtypes of the family overlays526, 524 and 522 are conveyed by the different respective text labels523, 525 and 527.

Business relationship overlay 530 is distinguished as to type from thefriends and family overlays 520/526/524 via use of a different, third“star” frame shape 505. The specific subtype of the businessrelationship linking the individuals 508 and 514, “Broker-Client,” isconveyed by the text label 529.

The present aspect also provides for positive indications of a failureto find a relationship. This may be inferred from depicting no overlayframe element or connection to the individuals 516 and 518. Or it may bepositively called out via use of the “No relationship found” overlayline 532, which uses another, different (fourth) “X” frame shape 507.The specific explanation or failure of the relationship determination isconveyed by the text label 531.

In some aspects the user can select any of the overlay relationshiplinks 520/522/524/526/530/532 and provide social network feedback, forexample a “Like” or a comment (“newly married in 2015”), etc.

The display of the overlay relationship links 520/522/524/526/530/532may be filtered in response to security settings or context definitionsfor relationships to be displayed. For example, the user may make aselection from a “Show Relationships” drop-down menu choice in agraphical user interface to display or specifically exclude any one orall of the family, business, friends links, as well as other links notshown in this example (LINKEDIN contact, sporting team teammate, etc.),in some examples as a function of the viewing status of the otherviewer. Thus, a selection of “show family relationships to family” by auser viewing or posting the photograph 502 view as depicted by FIG. 5may enable family members viewing the same photograph 502 via the sameor another social networking site to see the family relationshipoverlays 522, 524 and 526, but not the business relationship overlay 530or the Friends overly 520. A selection of “show all to all” will allowany user to see all of the overlay relationship links520/522/524/526/530/532. A selection of “show business overlays only tobusiness clients” may restrict the display of the Broker-Client overlay530 to only those users linked to the first user as a business contact(for example, via LINKEDIN, GOOGLE PLUS, FACEBOOK, etc.).

Thus, aspects enable user to determine and post a wide variety ofrelationships between the individuals 504, 506, 508, 510, 512, 514, 516and 518, each of which may be selectively displayed to other users basedon their status relative to the user.

The terminology used herein is for describing particular aspects onlyand is not intended to be limiting of the invention. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “include” and “including” when usedin this specification specify the presence of stated features, integers,steps, operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof. Certainexamples and elements described in the present specification, includingin the claims and as illustrated in the figures, may be distinguished orotherwise identified from others by unique adjectives (e.g. a “first”element distinguished from another “second” or “third” of a plurality ofelements, a “primary” distinguished from a “secondary” one or “another”item, etc.) Such identifying adjectives are generally used to reduceconfusion or uncertainty, and are not to be construed to limit theclaims to any specific illustrated element or embodiment, or to implyany precedence, ordering or ranking of any claim elements, limitationsor process steps.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A computer-implemented method for differentiallydriving a graphic display of links overlaid between people in a groupphotograph as a function of associated relationship type, the methodcomprising executing on a computer processor the steps of: identifyingeach of a plurality of different individuals that are each visiblewithin a photograph and distinguished from other ones of the visibleindividuals; identifying relationships among the identified individualsby extracting relationship data via natural language processing relativeto social network data of at least one of the identified individuals;and in response to determining that a plurality of the identifiedrelationships differ from one another with respect to a type ofrelationship, driving a graphic display device to display the identifiedplurality of relationships via each of a plurality of different overlayelements that link respective ones of the identified individuals withinthe photograph and each comprise labels indicating respective ones ofthe types of relationship identified for the linked individuals, whereineach of the different overlay elements use different graphic elementswith respect to others of the different overlay elements to conveydifferences in the respective ones of the types of the relationshipsthat are determined among the identified individuals relative to theothers of the different overlay elements.
 2. The method of claim 1,wherein the social network data comprises business association data,family data and social network contact data; and wherein the type ofrelationship is selected from the group consisting of family, friend andbusiness contact relationships.
 3. The method of claim 1, wherein thedifferent overlay elements are visually distinguished from one anotherbased on differences in respective ones of the types of relationship byusing different visual elements that are chosen from the groupconsisting of different colors, different fonts, different line weights,and different framing devices of the overlay elements that linkrespective ones of the identified individuals.
 4. The method of claim 1,further comprising: in response to determining an absence of any of theplurality of the different relationships among first and second ones ofthe identified individuals, driving the graphic display device todisplay another overlay element that is depicted directly on thephotograph and links the first and second ones of the identifiedindividuals, and comprise a specific label indicating that none of theplurality of the different relationships links the first and second onesof the identified individuals, and uses another different graphicelement that is visually distinguished from each of the plurality ofdifferent overlay elements to thereby convey that the step ofidentifying the relationships among the identified individualsdetermined no relationship among the first and second ones of theidentified individuals.
 5. The method of claim 1, further comprising: inresponse to determining that a relationship of a viewer of thephotograph on a social networking website via the graphic display deviceto a poster of the photograph on the social networking website is one ofthe types of the relationship selected from the plurality of differentrelationships, filtering the displayed overlay elements to thereby drivethe graphic display device to display only those of the plurality ofdifferent overlay elements that are associated with the determinedrelationship of the viewer to the poster.
 6. The method of claim 1,further comprising: integrating computer-readable program code into acomputer system comprising the processor, a computer readable memory incircuit communication with the processor, and a computer readablestorage medium in circuit communication with the processor; and whereinthe processor executes program code instructions stored on thecomputer-readable storage medium via the computer readable memory andthereby performs the steps of identifying each of the plurality ofdifferent individuals that are each visible within the photograph anddistinguished from other ones of the visible individuals, identifyingthe relationships among the identified individuals that differ from oneanother with respect to the type of relationship, and driving thegraphic display device to display the identified plurality of therelationships via each of the plurality of different overlay elementsthat are depicted directly on the photograph in response to determiningthat a plurality of the identified relationships differ from one anotherwith respect to a type of relationship.
 7. The method of claim 6,wherein the computer-readable program code is provided as a service in acloud environment.
 8. A system, comprising: a processor; a computerreadable memory in circuit communication with the processor; and acomputer readable storage medium in circuit communication with theprocessor; wherein the processor executes program instructions stored onthe computer-readable storage medium via the computer readable memoryand thereby: identifies each of a plurality of different individualsthat are each visible within a photograph and distinguished from otherones of the visible individuals; identifies relationships among theidentified individuals by extracting relationship data via naturallanguage processing relative to social network data of at least one ofthe identified individuals; and in response to determining that aplurality of the identified relationships differ from one another withrespect to a type of relationship, drives a graphic display device todisplay the identified plurality of relationships via each of aplurality of different overlay elements that link respective ones of theidentified individuals within the photograph and each comprise labelsindicating respective ones of the types of relationship identified forthe linked individuals, wherein each of the different overlay elementsuse different graphic elements with respect to others of the differentoverlay elements to convey differences in the respective ones of thetypes of the relationships that are determined among the identifiedindividuals relative to the others of the different overlay elements. 9.The system of claim 8, wherein the social network data comprisesbusiness association data, family data and social network contact data;and wherein the type of relationship is selected from the groupconsisting of family, friend and business contact relationships.
 10. Thesystem of claim 8, wherein the different overlay elements are visuallydistinguished from one another based on differences in respective onesof the types of relationship by using different visual elements that arechosen from the group consisting of different colors, different fonts,different line weights, and different framing devices of the overlayelements that link respective ones of the identified individuals. 11.The system of claim 8, wherein the processor executes the programinstructions stored on the computer-readable storage medium via thecomputer readable memory and thereby, in response to determining anabsence of any of the plurality of the different relationships amongfirst and second ones of the identified individuals, drives the graphicdisplay device to display another overlay element that is depicteddirectly on the photograph and links the first and second ones of theidentified individuals, and comprise a specific label indicating thatnone of the plurality of the different relationships links the first andsecond ones of the identified individuals, and uses another differentgraphic element that is visually distinguished from each of theplurality of different overlay elements to thereby convey that the stepof identifying the relationships among the identified individualsdetermined no relationship among the first and second ones of theidentified individuals.
 12. The system of claim 8, wherein the processorexecutes the program instructions stored on the computer-readablestorage medium via the computer readable memory and thereby, in responseto determining that a relationship of a viewer of the photograph on asocial networking website via the graphic display device to a poster ofthe photograph on the social networking website is one of the types ofthe relationship selected from the plurality of different relationships,filters the displayed overlay elements to thereby drive the graphicdisplay device to display only those of the plurality of differentoverlay elements that are associated with the determined relationship ofthe viewer to the poster.
 13. The system of claim 8, wherein theplurality of social network data of the at least one of the identifiedindividuals further comprises at least one of: a roster of a sport teamthat is relevant to a membership, activity or recreational activity ofthe at least one of the identified individuals; and a geographiclocation data of the photograph that is associated with a sporting eventthat includes the sport team as a participant.
 14. The system of claim8, wherein software comprising the program instructions is provided as aservice in a cloud environment.
 15. A computer program product forgenerating an interactive plotted electronic map presentation whichdepicts historical progression of events of a news story incident as afunction of topic, the computer program product comprising: a computerreadable storage medium having computer readable program code embodiedtherewith, wherein the computer readable storage medium is not atransitory signal per se, the computer readable program code comprisinginstructions for execution by a processor that cause the processor to:identify each of a plurality of different individuals that are eachvisible within a photograph and distinguished from other ones of thevisible individuals; identify relationships among the identifiedindividuals by extracting relationship data via natural languageprocessing relative to social network data of at least one of theidentified individuals; and in response to determining that a pluralityof the identified relationships differ from one another with respect toa type of relationship, drive a graphic display device to display theidentified plurality of relationships via each of a plurality ofdifferent overlay elements that link respective ones of the identifiedindividuals within the photograph and each comprise labels indicatingrespective ones of the types of relationship identified for the linkedindividuals, wherein each of the different overlay elements usedifferent graphic elements with respect to others of the differentoverlay elements to convey differences in the respective ones of thetypes of the relationships that are determined among the identifiedindividuals relative to the others of the different overlay elements.16. The computer program product of claim 15, wherein the social networkdata comprises business association data, family data and social networkcontact data; and wherein the type of relationship is selected from thegroup consisting of family, friend and business contact relationships.17. The computer program product of claim 15, wherein the differentoverlay elements are visually distinguished from one another based ondifferences in respective ones of the types of relationship by usingdifferent visual elements that are chosen from the group consisting ofdifferent colors, different fonts, different line weights, and differentframing devices of the overlay elements that link respective ones of theidentified individuals.
 18. The computer program product of claim 15,wherein the computer readable program code instructions for execution bythe processor further cause the processor to, in response to determiningan absence of any of the plurality of the different relationships amongfirst and second ones of the identified individuals, drive the graphicdisplay device to display another overlay element that is depicteddirectly on the photograph and links the first and second ones of theidentified individuals, and comprise a specific label indicating thatnone of the plurality of the different relationships links the first andsecond ones of the identified individuals, and uses another differentgraphic element that is visually distinguished from each of theplurality of different overlay elements to thereby convey that the stepof identifying the relationships among the identified individualsdetermined no relationship among the first and second ones of theidentified individuals.
 19. The computer program product of claim 15,wherein the computer readable program code instructions for execution bythe processor further cause the processor to distinguish each of theplurality of the overlay elements from one another based on type ofrelationship by using different visual elements for each of theplurality of the overlay elements that are chosen from the groupconsisting of different colors, different fonts, different line weights,and different framing devices of the overlay elements that linkrespective ones of the identified individuals.
 20. The computer programproduct of claim 15, wherein the computer readable program codeinstructions for execution by the processor further cause the processorto, in response to determining that a relationship of a viewer of thephotograph on the social networking website via the graphic displaydevice to a poster of the photograph on the social networking website isone of the types of the relationship selected from the plurality ofdifferent relationships, filter the displayed overlay elements tothereby drive the graphic display device to display only those of theplurality of different overlay elements that are associated with thedetermined relationship of the viewer to the poster.